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motion image data a portion of the image is lost and 
Image quality deteriorates even after correction. When 
motion compensation prediction is used the image qual- 
ity of suk>sequent pictures can also be adversely 
affected. This apparatus has a bit stream analysis por- 
tion (20) for analyzing each macro block of a moving 
image, a contaminated region registering portion (31) 
for registering a region as a contaminated region if an 
error has been detected during analysis, a motion vec- 
tor selection portion (14) for determining a predicted 
image to be used for motion compensation prediction, a 
contaminated region judging portion (16) for judging 
whether or not a predicted image is contaminated and a 
loop filter portk)n (20) for filtering a predicted image 
which is contaminated, so that it Is smoother for decod- 
ing. 
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Description 

BACKGROUND OF THE INVENTION 

5 FIELD QF THE INVgNTIQN 

The present invention relates generally to a motion image decoding method and an apparatus for decoding coded 
motion image data, and more particularly to a method of decoding coded motion image data using motion compensa- 
tion prediction and to a motion image decoding apparatus employing this method. 

10 

DESCRIPTION QF THE PRIOR ART 

Motion images generally have large levels of temporal and spatial redundance. In digital high efficiency coded tech- 
nology, redundancy is removed using such methods as motion compensation prediction and DOT (Discrete Cosine 

15 Transformation) and thereafter quantization is candied out and algorithms are often used for variabie-length coding of 
the quantization values and motion vectors to correspond with the generation frequency. Such algorithms are also used 
in international standard coding systems presaibed by ITU-TH.261 and MPEG (IS0/IEC1 1172-2. 13818-1). 

However in the event of some kind of malfunction during processing when sending, recording or reading data which 
has been coded with such algorithms, decoding errors can occur, such as. an inability to analyze variable-length codes 

20 or coded parameters with inconect values. Errors can lead to a partial loss of a decoded image. In the case where 
motion compensation prediction is being used, this loss can also adversely affect the decoded images which follow. 

In order to avoid this condition, decoding apparatuses often include an error concealment function which restores 
any lost image data with image data thoughrt to be similar to the lost image. Recently, there have been many proposals 
regarding en-or concealment algorithms. The majority of these are divided into those which spatially restore a lost por- 

25 tion using the data of closely positioned regions within a single picture (spatial concealment) and those which tempo- 
rally substitute a lost portion according to past or future image data which remains within the frame memory (temporal 
concealment). A picture is a unit of an image and is a concept which includes a frame and field. 

Spatial concealment implements the proposition that ''spatially adjacent small regions (pixels, blocks etc.) generally 
resemble one another (have a high correlation)," and, for example, in 'Performance of MPEG Codes in the Presence of 

30 Errors" (Y. Q. Zhang and X. Lee, 1 993) in SPIE Visual Communicattons and Image Processing '93, a macro block which 
has lost the infonnation for motion compensation prediction due to an.en^or (hereinafter referred to as a lost macro 
block") is replaced based on the average DC values of all the correctly decoded macro blocks in dose proximity to it. 

Temporal concealment implements the proposition that "motion between temporally close pictures is small (i.e. 
temporal correlation is high) and spatially adjacent regions move in a similar way." For instance, in 'Transmission Error 

35 Detection, Resynchronization and Error Concealment for MPEG Video Decoder" (S.H.Lee et al.. 1993) in SPIE Visual 
Communications and Image Processing '93, a method is demonstrated wherein the motion vector of a lost micro block 
is estimated from the motion vector of a conectly decoded block in close proximity, the estimated motion vector is used 
to determine a predicted image from past or future images and the lost micro block is sut}Stituted. 

These concealment technologies utilize general properties of motion images, and depending on tiie type of motion 

40 image, even if used independently, results are not always satisfactory. Methods have therefore been proposed for 
dynamically dealing with the properties of a variety of motion images and for switching between concealment processes 
as the case demarKls. For instance, in the technique proposed in "Adaptive Error Concealment Algorithm for MPEG 
Compressed Video" (H.Sun et al., 1992) in SPIE Visual Communications and Image Processing '92. tiie spatial and 
temporal correlation of blocks in close proximity to a block which is to be concealed are determined and based on these 

45 correlation ratios, switching is can-ied out between spatial concealment and temporal concealment. 

The above refers to decoding technology, but measures can also be taken to reduce the effects of errors in coding. 
In motion images, since propagation of the effect of loss in the temporal direction is particularly serious, closed intra- 
coding is often periodically carried out within a picture itself without carrying out motion condensation prediction, i.e. 
periodic refreshing is canried out. 

50 Fig. 1 is a figure depicting the conditions when coding is carried out using MPEG 1 or MPEG 2 and shows error 
propagation in the temporal direction generated when decoding. Here an I picture, which is tiie target of closed coding 
within the picture itself, is periodically inserted. As a result, even if loss occurs, when tiie following I picture arrives, tiie 
effect of tiie loss will not be visible and the error resistance of the motion image decoding sequence as a whole will be 
improved. In tiie case where tiiere is no I picture, the loss effect gradually spreads, as shown in Fig. 1 . In addition, there 

55 is anottier technique, known as intiB-slice. in which a group of macro blocks which have been intra-coded is periodically 
inserted. 

Since prediction coding such as DPCM (Differential Phase Code Modulation) is often carried out within a single pic- 
ture, DPCM includes a periodic reset unit so as to avert propagation of the effects of spatial loss. This unit corresponds 
for instance to ttie slice defined with MPEG 1 or MPEG 2 in Fig. 2. Rg. 2 also shows the condition of error propagation 
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in the temporal direction generated when decoding. It also shows cases in which an example macro block is comprised 
of 1 slice (A) and of 2 slices (B). In B. since the loss extent range is lessened when the length of the slice is shortened, 
it is possible to reset errors caused by DPCM and to decrease the propagation of the effect of spatial direction loss. 
Error concealment processing Is carried out for bit errors which cannot be corrected by error correction processing 

5 and its purpose is to limit image loss to a minimum. 

However, image data that has been concealed can still be unlike previous Image data, so that, in the subsequent 
picture, after carrying out motion compensation prediction based on a concealed image region, the predicted image is 
also unlike previous prediction images. Consequently temporal propagation of image quality deterioration occurs. Even 
when an I picture or an intra-slice is provided, errors can be generated when these are decoded. As such. I pictures and 

10 the like cannot be satisfactorily relied upon. 

The principal objective of previous decoding error processing has been how to carry out concealment processing 
as precisely as possibla The fundamental idea Is based on the fact that the image quality of a play-back image of a 
picture which has been processed is improved when the processing is more precise. However, In actual practice errors 
invariably arise in the concealment procesa Whenever motion compensation prediction is carried out, adverse effects 

15 resulting from these errors will spread. 

SUMMARY OF THE INVENTION 

It is therefore the object of the present Invention to restrict the spatial and temporal effects of errors which accom- 
20 pany concealment processing after confirming the existence of such eirors in concealment processing, while in order 
to improve the image quality of a play-k>ack image of a picture upon which that concealment processing has been per- 
formed. 

(1) The motion image decoding method of the present invention is a method for decoding coded motion image data 
25 using motion compensation prediction, including an analysis process for analyzing a motion image at each 

processing unit image (see below), a registering process for registering a region of a processing unit Image as a 
contaminated region when an error has been detected during analysis of a processing unit image, a determining 
process for determining a predicted image to be used for motion compensation prediction when a processing unit 
image is decoded, a judging process for judging if a predicted Image Is included in said contaminated region, and 

30 a snxx>thing process for smoothing a predicted image when sakJ predicted image has been judged to be included 
in said contaminated region. 

"Processing unit image" here refers to an image which is a processing unit when an image is decoded, for 
instance an MPEG macro block. Motion image data are coded in accordance with, for instance, ITU-TH. 261 , ISO 
/ lEC 1 1172-2 (i.e. MPEG 1). ISO / lEC 13818-2 (i.e. MPEG 2). "Analysis" refers to the reading of motion image 

35 data received, for instance, in a bit stream fornrtat. and is usually carried out at the first stage of decoding process- 
ing. In cases where motion image data includes an unreadable bit row, an error is detected. "Contaminated region" 
refers to a motion image region which suffers some type of adverse effect as a result of said en-or. "Predicted 
image" is an Image used In motion compensation prediction, for instance, when decoding a given macro block 
included in a given picture, when the region of the immediately preceding picture corresponding to the region of this 

40 macro block is identified, the region of the immediately preceding picture is equivalent to the region of the predicted 
Image of the macro block during decoding. 

With the above configuration, nrtotion image data is input and analyzed for every processing unit image. If no 
error is detected during analysis, decoding is carried out as usual. However if an enror is detected, the region of the 
processing unit image in which the en'or has been detected is registered as a contaminated region. 

45 Separate from this registering process, when a given processing unit image is decoded, a predicted image is 

determined for use in motion compensation prediction. Here, it is judged if the determined predicted image is 
Included in the said contaminated regfon, and if so. a snrmthing process Is performed on the prediction image. 
Thereafter, it is used as a predicted image of the processing unit inrmge during decoding and a decoded image is 
obtained. Filtering using a low-pass filter is an exanrple of such a smoothing process. 

50 Since a predicted image which is included in a contaminated region is smoothed, a state situation wherein the 

predicted image gives a distinctly.dissimilar impression from other portions of the image is avotdedr Since subse- 
quent pictures are decoded from this state.^ spatial or temporal deterioration in image quality is reduced: 

The above method, can also produce a visually satisfactory play-back image even if an uncorrectable bit error 
becomes mixed in. As a result, since there is no dependence on coded bit-stream formats, (formats stipulated by 

55 H. 261 . MPEG 1 , MPEG 2 and the like) the range of application is wide. 

(2) In one aspect of the present invention, the registering process registers both 1) a region of a processing unit 
image In which an error has been detected and 2) previous contaminated region for which motion compensation 
prediction has been carried out using a predicted Image as contaminated regions. The purpose of 1) is to note the 
existence or non-existence of an erra in a picture presently being decoded when decoding later picture while 2) 



3 



EP0 794 671 A2 



registers a warning for cases in which although no error has been detected in the picture presently being decoded, 
an error was registered as having occurred in a pre\/ious picture may have spread to the image which is currently 
being decoded. The warning is used when decoding subsequent pictures. By this method, the propagation of 
adverse effects due to an enror can be reduced. 

(3) The motion image decoding apparatus of the present invention is an apparatus for decoding coded motion 
image data using motion compensation prediction, including a bit stream analysis portion for analyzing each 
processing unit image of received motion image data, a contaminated region registering portion for registering a 
region of a processing unit image as a contaminated region when an error has been detected during analysis, a 
predicted image determining portion for determining a predicted image to be used for motion compensation predic- 
tion when a processing unit image is decoded, a contamination judging portion for judging if a predicted image is 
contaminated by comparing a region of a determined predicted image with said contaminated region, and a 
smoothing portion for smoothing a predicted image when said predicted image has been judged to be included in 
said contaminated region. The operation principles of this apparatus are as described in (1). 

Using this apparatus, received motion image data can be decoded and a play-back image obtained, with the 
added feature that visually satisfactory play-back image can now be obtained even if an uncorrectable bit error has 
become mixed in. 

(4) In one aspect of the present invention, the said predicted image determining portion carries out motion compen- 
sation prediction using the motion vector for each processing unit image and, where necessary, determines a pre- 
dicted Image for a processing unit image, which has lost its motion vector due to some error, based on the motion 
vector of a processing unit image which is in close proximity. As such, motion image data which has been lost can 
be restored. 

(5) In another aspect of the present invention, the present apparatus also includes a picture counting portion for 
counting tiie number of decoded pictures and when tiie number of pictures counted has reached a designated 
value, tiie said contaminated region registering portion de-registers the contaminated region. Since de-registering 
returns the state to one in which a region is assumed to be uncontaminated. the said smoothing processing is not 
caried out, and accordingly, excessive smoothing can be prevented. 

(6) In anotiier aspect of the present invention, the said contaminated region registering portion registers both 
processing unit images in whk;h an error has been detected, and also image regions for which motion compensa- 
tion prediction has been carried out using a predicted image in a previous contaminated region. The principle of 
tNs operation is the same as that explained in (2). As a result the propagation of adverse effects due to an error 
can be reduced. 

(7) In anotiier aspect of the present invention, tiie said contamination judging portion judges the existence of con- 
tamination from tiie level of overlap between the said predicted image and the said contaminated region. The pro- 
portion of the predicted intage within a contaminated region can be considered an example of "level of overlap." In 
other words, when the proportion of the predicted image within a contaminated region is high, this predicted image 
is easily judged to be contaminated. In this invention, since the standard used when judging a contaminated region 
can be set as tiie threshokl (TH) used in the judgment equation, a play-back image can be obtained tiiat is best- 
suited to the conditions. 

(8) In another aspect of the present invention, the relevant apparatus includes a loss image extent determining por- 
tion fbr determining at the time when an error has been detected, not only the processing unit image in which tiie 
error has been detected, but also the extent of the image region in which the processing unit image has lost its 
motion vector as a result of the error, and the said contaminated region registering portion registers this entire 
image region as a contaminated region. A contaminated region can tiuis be accurately tracked and deterioration of 
image quality can be reduced. 

(9) In another aspect of the present invention, tiie present apparatus includes a contaminated region memory por- 
tion for use with decoding pictures (decoding memory) and a contaminated region memory portion for use with pre- 
diction pictures (predicted memory), and if an error is detected in a picture presently being decoded, tiie said 
contaminated region registering portion registers the processing unit image in which the error has been detected 
in the contaminated region menx>ry portion fbr use with decoding pictures, and switches tiie contaminated region 
memory portion for use witii decoding pictures with ttie contaminated region memory portion for use with prediction 
pictures when the decoding; processing has proceeded to the next picture. "Decoding picture" refers to a picture u 
during decoding and "prediction picture" refers to a picture including in a prediction image:- 

Once a picture has been decoded, any error detected in the picture should be refen^ed to when the next picture 
is decoded. To this end. the contaminated region memory portion for use with decoding pictures is first switched to 
the contaminated region memory portion for use with prediction pictures. Since the original contaminated region 
memory portion for use with prediction pictures Is empty at tiiis point, it is switched to the contaminated region 
memory portion fbr use with decoding pictures. Thereafter tiiis switching is repeated each time a picture is 
decoded. As one exanrpte. a configuration is possible in which the contaminated region memory portion fbr use 
with decoding pictures Is used solely for registering, and the contaminated region memory portion for use with pre- 
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diction pictures Is used solely for reading. Thus by switching these as appropriate, registration and reference to 
contaminated regions is possible even with a small memory capacity. 

(10) In a final aspect of the present Invention, the present apparatus includes a picture configuration judging portion 
for judging whether the configuration of a picture is in frame format or in field format and a picture configuration 

5 converting portion for converting a picture configuration which is in frame format to a configuration in field format, 
so that when a picture configuration is in frame format, smoothing is carried out only after the configuration has 
been converted to field format. "Picture configuration" refers to the configuration of a picture of a processing unit 
image, including, at least, frame format and field format In this aspect, since frame formatted pictures are reorgan- 
ized into fields, motion existing between fields, which must not be smoothed, will not be smoothed. As a result a 

10 satisfactory image is obtained. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a figure showing coding as can'ied out using MPEQ 1 or MPEQ 2. and showing error propagation in the 
15 temporal direction generated during decoding. 

Fig. 2 is a figure showing a slice as defined by MPEG 1 or MPEG 2, and showing error propagation in the spatial 
direction generated during decoding. 

Fig. 3 is a figure depicting the data structure of an ITU-TH.261 video bit stream. 

Fig. 4 is a figure depicting the configuration of the decoding apparatus in a first embodiment. 
20 Fig. 5 is a flowchart showing a decoding procedure according to the apparatus of the first ennbodiment. 

Fig. 6 is a flowchart showing a concealment processing procedure. 

Fig. 7 is a diagram showing the internal configuration of the motion vector selection portion 14 of the first embodi- 
ment. 

Fig. 8 is a figure depicting an outline of the registering, judging and filtering processing for a contaminated region. 
25 Fig. 9 is a figure showing the internal configuration figure of the contaminated region judging portion 16 of the first 
embodiment. 

Fig. 10 is a figure depicting the state of processing of contaminated region registering and judging portion 31 . 
Fig. 1 1 is a figure showing the coefficients of a filter used in the loop filter portion 20. 
Fig. 12 is a configuration figure of the decoding apparatus in a second embocGment 
30 Fig. 1 3 is a figure depicting the data configuration of a video bit stream in compliance with the MPEG 1 video stand- 
ard. 

Fig. 14 is an internal configuration figure of a motion vector selection portion 14 in the second embodiment. 

Fig. 1 5 is a figure explaining the entire processing of registering, judging and filtering processing for a contaminated 
region according to the second embodiment. 
35 Fig. 16 is a figure depicting processing contents of a contaminated region registering and judging portion 31 for a 
B picture which has undergone interpolation prediction. 

Fig. 1 7 is a figure showing a judging method according to a contaminated region registering and judging portion 31 
in a case where 4 prediction images exist. 

Fig. 18 is an internal configuration figure of a loop filter portion 20 in a third embodiment. 

40 

DESCRIPTION OF THE P REFERRED EMBODIMENTS 
EmtpQciimgnt 1 

45 Fig. 3 is a figure depicting the data structure of an ITU-TH.261 video bit stream. A frame is positioned at a layer 
which is called the picture, and a GOB (Group of Blocks) layer is provided as a group of blocks. A macro block is com- 
prised of Y blocks used for brightness, a Cb block used for blue color-difference and a Or block used for red cok>r-dif- 
ference. The decoding apparatus in the present embodiment receives a coded bit stream in compliance with H.261. 
decodes it with macro block units and plays back the motion image data. Motion vector information obtained for each 

50 macro block by carrying out compensation prediction between pictures is used when decoding. 

It is a characteristic of the processing of the present apparatus that when a picture is decoded, a macro, block 
region which is a target of concealment processing is registered as a contaminated region, and thereafter if a predicted 
image of a macro block enters this contaminated region when other pictures are decoded, filtering processing is carried 
out for this prediction image. 

55 

[Configuration] 

Fig. 4 is a figure depicting a configuration of the decoding apparatus in a first embodiment. This apparatus first 
receives a vkieo bit stream 1. This bit stream is a coded data stream formed from image infbrmatk>n and additional 
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information. Image information is obtained by quantizing at each block a transformation coefficient obtained from the 
orthogonal transformation of the original image or the motion compensation prediction error image data of every macro 
block and then carrying out variable-tength coding of a quantization index. Additional information is added at each pic- 
ture and at each macro block The present apparatus has a configuration which will follow this coding process in 
5 Inverse. 

This apparatus includes a variable-length decoding portion 2 for decoding a received video bit stream 1 , a inverse 
quantization portion 5 for carrying out inverse quantization using a quantization DOT coefficient 3 and a quantization 
parameter 4 received from the variable-length decoding portion 2. a inverse DOT portion 6 for receiving the inverse 
quantumization result from inverse quantization portion 5 and carrying out an inverse OCT computation, a motion vec- 

10 tor selection portion 14 for selecting and outputting a motion vector which will most accurately express the motion of a 
macro block (hereinafter referred to as a linal motion vector") based on screen position information 11 and a motion 
vector 12 of the relevant macro block, output from the variable-length decoding portion 2, a contaminated region judging 
portion 16 for judging if a predicted image is contaminated after refemng to a final motion vector and an error flag 8 
described below, a frame memory 1 9 for storing decoded image data, a memory interface portion 1 8 for controlling the 

IS writing and reading of frame memory 19, an OR gate 40 for notifying fitter 20 when eitiier of a loop filter operation 
instruction flag 10 or a contaminated region filtering instruction flag 17 described below has reached 2, a loop filter 20 
which is a low pass filter in compliance with ITU-TH.261 for carrying out filtering of image data read from frame memory 
19 when notification is received from OR gate 40, and a decoding addition portion 7 for calculating a predicted image 
from the output of loop filter 20 and the output of Inverse DOT portion 6 and generating a final decoded image 22. 

20 The variable-length decoding portion 2 of this embodiment has a bit stream analysis portion 2Q0 for analyzing 
received bit stream 1 , a loss image extent determining portion 204 for determining an image region which has been lost 
due to en-or when an error has been detected by ttie result of analysis, and a picture counting portion 208 for counting 
the numt)er of decoded pictures in order to determine the contaminated region tracking period, desaibed below. The 
picture counting portion 208 may also count the number of pictures after an error has been detected. In either case, 

2S picture counting portion 208 issues a contaminated region storage memory reset instruction 1 3. described below, to the 
contaminated region judging portion 16 once the count number has reached a designated value. 

[Operation] 

30 Fig. 5 is a flowchart showing a decoding procedure according to an apparatus of tiie first embodiment. Bit stream 
analysis portion 200 analyses data stream 1 t>ased on the H.261 syntax and a judgment is made concerning the exist- 
ence of an error (S100). Here, an error is detected which does not conform to the specified code language. According 
to the decision at SI 00, tiie process branches depending on whether or not an en'or is found if no error is detected at 
decision SI 06 the process branches depending on whether tiie image being processed has been intra-coded or inter- 

35 coded. IntiB-coding refers to coding which is carried out without including motion vector information, and inter-coding 
refers to coding in whfoh motion vector information is included in the attribute information at the time of coding. 

[1] Normal Decoding Operation 

40 If no error is detected at 8100, tiie decoding process commences as usual. In the case where the target of the 
processing is a macro block which has been intra-coded (hereinafter referred to as an Inf a-coded macro block"), that 
is to say in a case where the intra / inter flag 9 is indicating intra, processing passes from the length-variable decoding 
portion 2 to the inverse quantization portion 5. and then the inverse DOT portion 6. Since intra-coded maao t^locks do 
not include motion vector information in principle a configuration for motion vectors is not used. The output of the 

45 inverse DOT portion 6 presents an original signal within a frame and this passes through the decoding addition portion 
7 to become the final decoded image 22. In Fig. 5 this processing is shown as a path in which inverse quantization 
(8102) and inverse DOT (8104) are canied out and a decoded image is output (8108) following a Judgment as to 
whetiier or not it is intra-coded (8106). 

In tiie case where the target of the processing is a macro t}lock which has been inter-coded, that is. where tiie intra 

50 I inter flag 9 indicates inter, tiie signal is processed by both the path described above and a patii passing through a 
motion vector selection portion 1 4, and tiie processing results of tiiese are calculated by a decoding calculating portion 

The former process is carried out by tiie Inverse quantization (Si 02) and inverse DOT (8104) shown in Fig. 5. In 
this case, tiie output signal of tiie inverse DOT portion 6, is also a signal for predicting eror between motion compen- 
55 sation prediction frames. The latter process con-esponds to 81 10-122 in Fig. 5. Since tiie processing target in 8106 is 
an interceded macro block, the processing proceeds to 81 10. A final motion vector 15 is then output from ttie motion 
vector selection portion 1 4. If ttiere is no error, tiie motion vector of the relevant macro block can simply be used as the 
final motion vector. Thereafter the on-screen position information 1 1 and the final motion vector 15 are sent to the mem- 
ory interfece portion 18, the address of the predicted image is produced and the predicted Image is extracted from the 
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frame memory 19 (S1 12). The predicted image is then sent to the loop filter portion 20. 

The existence of contamination in the predicted image is judged in parallel with the above process (S1 14). A 
detailed description of the judging method is given below. If it is determined that the iniage is contaminated (Y in S11 4). 
the on-screen position information for that predicted image is registered in a decoded picture contaminated region 

5 memory 36 described below (S1 16). This is so that this information can be refen-ed to when subsequent pictures are 
decoded. N ext the contaminated region filtering instruction flag 17 is set to 1. As a result, filtering is performed at the 
loop filter portion 20 (S1 18) and a final state 21 of the predicted image(hereinafter referred to as linal predicted image 
21 ") is obtained. Furthermore, since judgment of the existence of contamination at Si 14 is possible by referring to the 
decoded picture contaminated region memory 36, extraction of the predicted image(S1 12) does not have to be earned 

10 out before Si 1 4, and these processes can be performed in parallel. 

The final predicted image 21 is sent to the decoding addition portion 7, added to the output of the inverse DCT por- 
tion 6 and a decoded image 22 is output (SI 20). Since the decoded image 22 will be used as a precficted image for 
subsequent pictures, it is written in the frame memory 19 based on the on-saeen position information 1 1 of the relevant 
macro block. 

IS If a predicted image is judged not to t^e contaminated in S1 14 (N in S1 14) filtering is performed only in the case in 
which the loop filter operation instruction flag 10 is at 1 (S122). In the H.261 specification, the loop filter operation 
Instruction flag 10 is a bit flag to be set for each with maao block unit in the original bit stream 1 and the value of the 
flag depends on the coding side. 

20 [2] Decoding Operation when En'or is Detected 

(1) Concealment Processing 

When an error is detected atSlOO, concealment processing is first carried out (S124). Errors can occur irrespective 
25 of whether a macro block is intra-coded or inter-coded. Concealment processing refers to the process in which a motion 
vector is estimated for a macro block which has been lost due to error and a substituted image is extracted from a pic- 
ture which has already been decoded and is being stored in the frame memory 1 9 (since such a picture will be used for 
prediction. It is hereinafter referred to as a "prediction picture'^. S ince this predicted picture will be used as a predicted 
for the relevant maao block when an error is detected, the output of the Inverse DCT portion 6 is ignored. 
30 Fig. 6 is a flowchart showing procedures for concealment processing. As the flowchart shows, a loss image extent 
determining portion 204 first identifies the extent of Image data which has suffered adverse effects due to the error 
(S200). For instance, in the case where an error has been generated in a picture layer and continuity of decoding is not 
possible (Y in S202), all data relating to that picture is discarded (S220) and concealment processing is discontinued, 
tn such a case, since one picture disappears, another process is carried out such as one which displays the same pic- 
35 ture twice. 

Alternatively, for N at S202, because it is assumed that an error has been generated for which continuity of decod- 
ing is not possible at a level below the GOB layer, an error flag 8 is first set to 1 (S204} and the data relating to the GOB 
in which the en^or occun^ed is discarded (S206). By this process, data for a number of macro blocks are lost. A motion 
vector is estimated for these macro blocks (hereinafter referred to as "lost macro blocks") (S208). Next, following an 

40 evaluation of the adequacy of tiie estimated motion vector (S210). a final motion vector is output (S212). 

Processing from the estimation of a motion vector to the output of a final motion vector is can-ied out in tiie motion 
vector selection portion 14. Fig. 7 is an internal configuration figure of the motion vector selection portion 14 of the first 
embodiment. The motion vector selection portion 1 4 has a motion vector buffer 26 which stories a motion vector for 
each macro block in a plurality of macro block lines, a writing vector determining portion 23 which refers to error flag 8 

45 and intra / Inter flag 9 and supplies a writing Instruction 24 and a writing vector value 25 to the motion vector buffer 26 
to cont'd writing, a reading vector determining portion 28 which supplies a reading instruction 29 to tiie motion vector 
buffer 26 and controls reading of the vector value, and a final motion vector determining portion 30, into which the esti- 
mated motion vector 27 is input from the niK>tion vector buffer 26 In order to judge the adequacy of that vector, which 
and outputs a final motion vector 15. The final motion vector determining portion 30 also makes reference to error flag 

so 8 and the on-screen position information 11 for the relevant macro block. 

In the present embodiment the motion vector for amacro tslock immediately atx>ve the lost macro block is. estimated— iTtci 
as the motion vector of the lost macro block. This is based on the experience that motion vectors between proximate ' ' 
macro blocks are highly similar. In this process, It is sufficient for the motion vector buffer 26 to maintain a motion vector 
which relates to a macro block included in a macro block line which is one line above the macro block line in which tiie 

55 lost macro block exists. 

In ttiis configuration, while the error flag 8 is at 0. the writing vector determirdng portion 23 writes 0. for an intra- 
coded macro block, or a decoded motion vector value, for an inter-coded macro blocK as ttie writing vector values in 
the motion vector buffer 26. Alternatively, while the error flag 8 is at 1. since no accurate motion vector exists for the 
relevant macro block itself. 0 is written as the vector value 25. This is a precaution taken so tiiat the macro block direcfly 
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below does not suffer any adverse effect from the motion vector of the current lost macro block in the subsequent 
processing. 

When the error flag 8 is at 1 , based on the on-screen position information 1 1 of the lost macro blocK the reading 
vector determining portion 28 reads the motion vector of a macro block immediately above the cunrent lost macro block 

5 as an estimated motion vector 27. When the enror flag 8 is at 0, since the motion vector of the relevant macro k)lock is 
accurately decoded, it is read as is. 

When the error flag 8 is at 1 . the adequacy of the estimated motion vector 27 is evaluated at the final motion vector 
determining portion 30 (S21 0). In the present embodiment, when the estimated motion vector 27 is used as the motion 
vector for the relevant macro block, if the vector points outside of the saeen. this is judged to be inadequate. In such a 

10 case the vector value is set to 0 and output as a final motion vector 15. If the estimated motion vector 27 is adequate, 
it is output as a final motion vector 1 5 (S21 2). TTie final motion vector 1 5 is transferred to the memory interface portion 
18 and the substituted image obtained (S214) is taken as the prediction Imaga Thereafter the error flag 8 is returned 
to 0 (S216), and the process proceeds to the next QOB (S21 4) and the concealment processing ends. 

When the concealment processing has ended, the macro block which has been the object of the concealment 

15 processing is registered in the decoded picture contaminated region memory 36 as a contaminated region (SI 26). In 
due course, the predicted image obtained is output as a decoded image 22 (SI 08). In the present emtxxliment, when 
the error flag 8 is at 1 , the output of the inverse OCT portion 6 is ignored at the decoding addition portion 7 irrespective 
of whether it is for an intra-coded macro block or an inter-coded macro block. As a result the predicted image deter- 
mined from the estimated motion vector 27 becomes the decoded image. In addition, since addition of prediction en'or 

20 signals between compensation prediction frames is not can'ied out when a lost macro block is decoded, the fatering 
described below is not carried out. 

(2) Judging and RItering of Contaminated Reigions 

25 The processes of judging of contaminated regions at 81 1 4. registering of contaminated regions at S116 and 8126. 
and filtering at S1 18 will now be explained. 

(2-1) Outline 

30 Fig. 8 is a figure depicting an outline of these processes. Rrst in a picture 80. in which an en^or has been detected, 
a group of macro blocks including the macro block 83 in which an error has been detected, and for which concealment 
processing has been can-ied out. is registered as a contaminated region 84. In the next picture 81 , even in places where 
no enror has been detected in bit stream 1 , as with macro blocks 85 and 86 in this figure, motion compensation predic- 
tion can be carried out from prediction images 87 and 88, which have at least one portion included in the contaminated 

35 region 84. Since the desired ideal predicted image for these macro blocks 85 and 86 is not obtained, the image quality 
of the decoded image deteriorates. Similarly, since motion compensation prediction is carried out based on the predic- 
tion images 91 and 92 which are included In the contaminated region 90 of picture 81, the contamination spreads to 
macro blocks 93 and 94 of the picture 82. Since motion compensation prediction is here earned out in maao block 
units, the image which is output has distinctive macro block borders. RItering is therefore applied to prediction images 

40 included in the contaminated region, the decoded image is smoothed to a certain extent and the distlnctivity of the 
macro block borders is decreased. As a result, adverse effects of contamination can also be reduced in a case where 
a region to which filtering processing has been applied is to be a predicted image for subsequent pictures. 

The judging of contaminated regions, registering of contaminated regions and f ifterlng process are canried out at 
the necessary places (3, 5-7 below) during the following sequence: 

45 

1 . an error is detected when a certain picture is decoded 

2. concealment is candied out for the lost maao block 

3. the region of said lost macro block is registered as a contaminated region 

4. when decoding another picture, a predicted image of a certain macro block is used 
so 5. it is determined that a predicted image is included in the contaminated region 

6. the region of that predicted Jmage is re-registered as a contaminated region • 

7. filtering processing is applied to that predicted image . . j 

8. a decoded image is output based on the filtered predicted image 

55 SI 26 in Fig. 5 corresponds to 3 atxsve and is characterized In that it is carried out without a contamination judg- 
ment. However. S1 16 corresponds to 6. above and is carried out with reference to the contamination judgment result. 
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(2-2) Judgment and Registration of a Contaminated Region 

Judgment of a contaminated region is carried out at a contaminated region judging portion 16. Fig. 9 is an internal 
configuration figure of a contaminated region judging portion 16 of the first emkxxliment. 

5 The contaminated region judging portion 16 uses the error flag 8. the on-screen position information 11 of the 
macro blocK the final motion vector 15, and a contaminated region memory reset instruction 13 (to be described 
below), and includes a contaminated region registering and judging portion 31 , for controlling reference to registration 
of the contaminated region, and outputting a contaminated region filtering instruction flag 1 7. The contaminated region 
judging portion 16 also includes a contaminated region memory 37 for use with decoded pictures and a contaminated 

10 region memory for use with prediction pictures 37. The former registers a contaminated region included in a picture 
which is presently being decoded and the latter is referred to check whether or not a predicted image is included in a 
contaminated region. When the decoding of one picture is completed, the contaminated region memory for use with 
decoded pictures 36 is switched to the contaminated region memory for use with prediction pictures 37 so that any reg- 
istered contaminated regions can be referred to when decoding the next picture. Since the contaminated region mem- 

15 ory for use with decoded pictures 36 is now empty, it is used as the contaminated region memory for use with prediction 
pictures 37. Thereafter each time the decoding of one picture is completed the contaminated region memory for use 
with decoding pictures 36 and the contaminated region memory for use with prediction pictures 37 are switched. The 
former memory is used only for registering, the latter memory is used only for reference. 

The contaminated region registering and judging portion 31 controls the contaminated region memory for use with 

20 decoding pictures 36 with a writing instruction 32 and an address 33 (included in the on-screen position information 1 1 
of the macro block presently being processed), and similarly controls the contaminated region memory for use with pre- 
diction pictures 37 with a reading instruction 34 and an address 35 which is the target of the reading. 
The operation of the above configuration will now be explained. 

When the error flag 8 is at 1 , tiie contaminated region registering and judging portion 31 regards the macro block 
25 presently being processed as contaminated and registers the on-screen position information 1 1 of that macro block with 
the contaminated region memory for use with decoding pictures 36. This corresponds to SI 26 in Fig. 5. 

Alternatively, if the error flag 8 is at 0. contamination judgment is first can-ied out. Fig. 10 is a figure depicting the 
state of processing in the contaminated region registering and judging portion 31 . In this figure a predicted image region 
140 is first determined from the final motk>n vector 15 and the on-saeen position information 1 1 of the relevant macro 
30 block. Next a macro block address within the predicted picture and having a region overlapping the predicted image 
region 140 is obtained. Based on this address the distribution of tiie contaminated region 38 shown in the figure is 
obtained from the contaminated region memory for use with prediction pictures 37. An contamination is made based on 
the degree of overlap between ttiis contaminated region 38 and the predicted image region 140 using tiie following 
equation: 

35 

P1 > TH (Equation 1) 

PI is the share of the contaminated region included in tiie prediction imaga For example, if the area of tiie portion 
of the predicted image region not overlapping witti tiie contaminated region is SI and tiie area of the overlapping por- 
40 tion is S2, then a calculation Is carried out with: 

PI =S2/(S1 +S2) 

TH is the threshold value for determining a contaminated region. In the present embodiment a predicted image is 
45 judged to be contaminated if it satisfies this equation. TH in this equation is set independent of tiie properties and con- 
tent of the motion image and the decoding conditions • a fixed value may be used or the value can be varied in response 
to conditions. According to the setting of this value, the distoibution aspect of the contaminated region can be controlled 
and image control is possible in response to decoding conditions. 

If tiie result of the judgment is ttiat a predicted image of a macro block being decoded is included in a contaminated 
50 region, tiie on-screen position information 1 1 of the relevant macro block is registered as a contaminated region witii 

the contaminated region memory for use witii decoding pictures 36 and the contaminated region filtering instruction flag <r.ci 
1 7 is set to 1. ' . ; 

Here the question has been that of whether or not a contaminated region exists in a prediction picture, but in a case 
in which an error is detected in, for instance, a picture being decoded, the error flag 8 is set to 1. the concealment 
55 processing desaibed above is carried out until immediately before tiie next GOB and the macro blocks which have 
been the targets of the processing are all registered as contaminated regions. 
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(2*3) Filtering Process 

When the contaminated region filtering instruction flag 1 7 is at 1 . a loop filter portion 20 is operated and a low-pass 
filter is applied to the prediction image. Fig. 11 is a figure showing the coefficients for a possik^le filter to be used in the 
5 loop filter portion 20. The numbers in the figure indicate the filter coefficients for a pixel at O. In the present embodiment 
a filter in compliance with H.261 is used as this filter. Filtering is applied to all blocks formed from 8x8 pixels which 
define brightness and color differences. 

(2-4) Limitation of RKering Process 

10 

RItering over a long time period leads to deterioration of the resolution. In the present embodiment see Fig. 4, a 
picture counting portion 208 counts the number of pictures decoded after error detection and issues a contaminated 
region memory reset instruction 1 3 to the contaminated region judging portion 16 when the count value has reached a 
designated value. 

IS The contaminated region registering and judging portion 31 complies with tiie contaminated region memory reset 
instruction 13 and according to a reset instruction 39 carries out initialization of the contents of the contaminated region 
memories for use with prediction pictures and decoding pictures 36 and 37, which erases all previous registrations. As 
a result, contaminated region tracking is halted, contaminated region registration and filtering is not carried out until the 
next error is detected, and excessive smoothing due to filtering Is prevented (the interval between resets of the oontam- 

20 inated region memory is hereinafter refen-ed to as a "contaminated region tracking period"). The determining method of 
the contaminated region tracking period Is settable, and either fixed or variable values may be used. According to this 
setting, tiie disti-ibution aspect of tiie contaminated region can be controlled and image control is possible in response 
to decoding conditions. 

2S Embodiment 2 

[Configuration] 

Fig. 12 Is a configuration figure of a decoding apparatus In a second embodiment In this embodiment a bit stream 
30 which has been coded in compliance with MPEG 1 video (ISO / lEC 1 1 130-2) is received and decoded and motion 
image data are played back. 

Fig. 1 3 is a figure depicting the data configuration of a video bit stream which complies with MPEG 1 video. Frames 
are positioned in layers referred to as pictures and, based on the permitted prediction direction, tiiese picture are 
divided as: I pictures, P pictures and 6 pictures. A picture is a collection of slices and each slice is a collection of macro 

35 blocks. A macro block is formed from 4 brightness blocks Y1 ~4 and color blocks Cb and Cr. The arrows in the "Predic- 
tion Direction" section indicate the prediction direction. The arrows pointing to the right indicate fonward prediction and 
those pointing to the left indicate reanward prediction. For I pictures, motion compensation prediction is not canied out 
and closed coding (irrtra-coding) is carried out within the picture. For P pictures, only motion compensation prediction 
from temporally previous pictures (forward prediction) is permitted. For B pictures, motion compensation prediction from 

40 temporally later pictures (rearward prediction) is also permitted. In addition, prediction which combines prediction from 
temporally previous pictures with prediction from temporally later pictures is also possible and is termed interpolated 
prediction (bi-directional prediction). A slice layer is formed from a collection of macro blocks. A slice is the minimum 
decoding synchronous unit. 

The same codes are used for elements in Rg. 1 2 which correspond to those in Fig. 4 and only those portions which 
45 differ from the first embodiment will be explained. 

Fig. 12 differs from Fig. 4 in tiiat: ttie receiving video bit sti'eam 1 is MPEG 1 video-compliant; a picture configura- 
tion judging portion 206. which judges the picture configuration, is added in the variable-length decoding portion 2 (how- 
ever, tiie picture configuration judging portion 206 is not itself required for MPEG 1 as will be explained in relation to 
MPEG 2 in a third embodiment): quantization parameter 3. quantization OCT coefficient 4, on-screen position Infbrma- 
50 tion 1 1 and motion vector 12 are sent via a parameter bus 42 to tiie inverse quantization portion 5; a loop filter 20 is not 
^ required and instead a. filter portion 56 is used, which operates in compliarK^e with a contaminated region. filtering ".. * 

instruction 1 7; loop filtering operation instruction 10 and OR gate 40 have been deleted along witti the deletion of loop : r - 

filter 20; and tiie prediction direction 51 used in decoding is supplied to the comaminated region judging portion 16. RN 
ter portion 56 executes the filtering process In compliance with the coefficients shown in Fig. 1 1 . 

55 

[Operation] 

Those operations which differ from those for tiie first embodment will now be explained. 
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[1] Normal Decoding Operation 

S100~122 in Fig. 5 are carried out. When the decoding target is an Intra-coded macro btocK the inverse quantiza- 
tion portion 5 extracts a quantization parameter and a quantization OCT coefficient from tiie parameter bus 42. There- 

5 after, the process is the same as in the first embodiment. 

When the decoding target is an inter-coded macro block, the motion vector selection portion 14 latches the motion 
vector and tiie prediction direction information sent tiirough tiie parameter bus 42 and outputs them as tiiey are as the 
final motion vector 15 and the final prediction direction information 51 respectively. A t the memory interface portion 1 8 
the type of the picture to which the relevant macro block belongs and the on-screen position information of the relevant 

10 macro block are latched from the parameter bus 42. A prediction picture is ttien determined by referring to the picture 
type and a picture address is created based on the on-screen position information, final motion vector 15 and final pre- 
diction direction information 51 of tiie relevant macro block, and a predicted image is extracted from the frame memory 
1 9. The predicted image is not put through the filter but is sent as is as the final predicted image 21 to decoding addition 
portion 7 where it is added to tiie output of inverse DOT computation portion 6 to become final predicted image 22. 

75 Since the decoded image of an I picture or B picture will be used as a prediction picture for subsequent pictures, it is 
written once again in the frame memory 19 based on the on-screen position information of the relevant macro block 
latched at the memory interface portion 18. 

[2] Decoding Operation when En-or is Detected 

20 

(1) Concealment Processing 

The processing shown in Fig. 6 is carried out. However, in the present embodiment, because GOBs are not used 
in MPEG, GOB should be interpreted as slices. The present embodiment is characterized in that extraction of a predic- 
ts tion image is carried out taking into consideration not only the motion vector but also the prediction direction. In this 
embodiment the following prediction pictures and motion vectors are used in concealment processing for each of tiie 
types of picture. 

1. {Pictures 

30 

Motion compensation prediction is not normally carried out for I pictures but the concept of a prediction picture is 
introduced. The most recentiy decoded I or P picture is taken as ttie prediction picture and the estimated motion vector 
is put at zero. In other words, a macro block at the same on-screen position as the macro block now being processed 
is taken unchanged from the most recentiy decoded 1 or P picture as the prediction image. Therefore it may be said that 
35 I picture concealment is caried out according to forward prediction. This prediction direction is treated as an estimated 
prediction direction, described below. 

2. P Pictures 

40 As in tiie first embodiment the motion vector of the macro block immediately above the lost macro block is taken as 
the estimated motion vector. The prediction picture of tiie macro block immediately above is also taken unchanged as 
the prediction picture. Therefore the prediction direction for P pictures is also fbnward. 

3. B Pictures 

45 

The movement vector follows the maao block immediately above as witti P pictures. Therefore, in B picture con- 
cealment, the estimated precfiction direction is either forward or reaiward depending on ttie prediction direction of tiie 
macro block immediately above tiie lost macro block. 

The process wherein a final motion vector 1 5 is created from an estimated motion vector thus determined is carried 
50 out by tiie motion vector selection portion 14. Fig. 14 is an internal configuration figure of tiie motion vector selection 
portion 1 4 in the second emkxxliment. The main.difference between this figure and the configuration shown in Fig. 7 is ' 
: that a parameter latch portion 59 is provided for latching a signal group on the parameter bus 42.TThe parameter latch '..^r. 
portion 59 latches the motion vector, prediction direction and on-screen position information of a macro block being 
decoded. The latched motion vector 60. prediction direction 61 and on-screen position information 62 are supplied to a 
55 writing vector determining portion 23 and a final motion vector determining portion 30. A further difference is that in 
addition to the estimated motion vector 27, ttie estimated prediction cfirection 69 desaibed above is also sent from 
motion vector buffer 26 to final motion vector determining portion 30. 

When the error flag 8 is at 1 , the motion vector and prediction direction of the macro block immediately above that 
macro block are extracted respectively from the motion vector buffer 26 as tiie estimated motion vector 27 and the esti- 
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mated prediction direction 69 based on the on-saeen position information of the macro block being decoded. These 
are output through the final motion vector determining portion 30 as the final motion vector 15 and the final prediction 
direction 51 . The final motion vector determining portion 30 evaluates the adequacy of the estimated motion vector 27 
based on the estimated motion vector 27 and the estimated prediction direction 69. When the result of the evaluation is 
5 that the estimated motion vector 27 is inadequate. 0 is output as the value of the final motion vector 15 and forward is 
output as the final prediction direction 51. These results are transferred to the memory interface portion 18 and there- 
after, using a process Identical to that in the first embodiment a predicted image is obtained. 

(2) Judging and Filtering of Contamination Regions 

10 

Fig. 1 5 Is a figure for explaining the registering, judging and filtering processing of a contaminated re^on according 
to the second embodiment, it depicts the state of a B picture which Is decoded according to interpolation prediction. 
Here, an error has been detected at a macro block 100 in the first picture 95 and the slice which includes that block has 
been registered as a contaminated region 101. Macro blocks 104 and 105 in the following P picture 97 have been 

15 decoded from the prediction images 102 and 103 included in the contaminated region 101 with the result that the con- 
taminated region 106 in the P picture 97 has expanded. The macro tHock 107 which is actually being decoded now in 
the B picture 96 is using the predicted image 108 included in the B picture 96 and the predicted image 109 included in 
the P picture 97. Thus, contamination is propagated. 

In MPEG. B pictures are not used for prediction of other pictures. Therefore no reference is made to a contaminated 

20 region in a B picture when decoding other pictures. Consequently registration of contaminated regions Is only carried 
out fbr I pictures or P pictures. In other words, in an I or P picture a macro block for which error concealment has been 
carried out is registered as a contaminated region. 

Judging and registering of a contaminated region is carried out by a contaminated region judging portion 16. The 
configuration of contaminated region judging portion 16 in the present embodiment is largely similar to that shown in 

25 Fig. 9. It differs a contaminated region registering and judging portion 31 also makes reference to final prediction direc- 
tion 15 and to parameter bus 42, and reading instruction 34 and address 35 are supplied not only to contaminated 
region memory for use with prediction pictures 37 but also to contaminated region memory for use with decoding pic- 
tures 36. Reference is made to the parameter bus 42 in order to obtain the on-saeen position information of the macro 
block being decoded. 

30 When the error flag 8 is at 1. the macro block being decoded is judged by the contaminated regk)n registering and 
judging portion 31 to be a contaminated region and Its on-screen positk)n information is registered the contaminated 
region memory for use with decoding pictures 36. Such registration is carried out only fbr I and P pictures. 

Alternatively, the contamination judging carried out when the error flag 8 is at 0 applies only to P and B pictures. An 
I picture makes no reference to other pictures, so there is no need to judge contamination for pictures which have been 

35 decoded. 

Processing by the contaminated region registering and judging portion 31 for P pictures is the same as that shown 
in Rg. 10. Two prediction images exist only in cases in which interpolation prediction is carried out at a B picture. Rg. 
16 is a figure depicting the processing of the contaminated region registering and judging portion 31 for a B picture 
which has been composed using interpolation prediction. In both the forward prediction picture and the rearward pre- 
40 diction picture the distribution of the contaminated region 38 is obtained as in the first embodiment. 81 -84 are here 
defined as follows: 

SI is the portion of the predicted image region 1 40 in the forward prediction picture which does not overlap with the 
contaminated region 38 of that picture 
45 ' S2 is the portion of the predicted image region 140 in the fonvard precfiction picture which overiaps with the con- 
taminated region 38 of that picture 

S3 is the portton of the predicted image regton 1 40 in the reanward predkition picture which does not overiap with 
the contaminated region 38 of that picture 
- S4 is the portion of the predicted Image region 1 40 In the reanward directional prediction picture which overiaps with 
so the contaminated region 38 of that picture 

- and thus.thecontafrHnatedregion.shareP2 within the predfeted image is - -:r-*s.j.... . 

P2 {82 / (81 + S2) ^> 84 / (S3 + S4) }/ 2 

55 

As in the first embodiment, when P2 satisfies the conditions below, that predicted Image is regarded as contaminated. 

P2 > TH (Equation 2) 
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Furthermore, in the case when a B picture is created with only forward prediction, P2 is 

P2 = S2/(S1+S2) 
5 and in the case of only reanivard prediction 

P2 = S4/(S3 + S4). 

Furthermore, since a B picture which has been composed based on interpolation prediction has two prediction 
10 images, the contaminated region memory for use with decoding pictures 36 and the contaminated region memory for 
use with precfiction pictures 37 are both used as contaminated region memories for fbnvard and reanvard prediction pic- 
tures. 

TTiereafter, filtering processing is the same as in the first embodiment. As in the first embodiment, the registration 
of the contaminated regions is also erased after each contaminated region traddng period. 

IS 

Embodiment 3 

A decoding apparatus for decoding and playing back a bit stream which has been coded in compliance with MPEG 
2 video stipulations will now be explained. This differs from MPEG 1 in that one mode can be selected from a plurality 
20 of motion compensation prediction modes when coding. 

[Configuration] 

The configuration of this apparatus is largely identical to that shown in Fig. 12 but differs in that a motion compen- 
25 sation prediction mode is supplied between the motion vector selection portion 1 4 and the contaminated region judging 
portion 16. 

The data configuration for a bit stream which complies with MPEG 2 video stipulations is the same as that shown 
in Fig. 1 3. A picture is defined as a frame or a field type and the distinction is termed the picture configuration, the former 
being a frame picture and the latter a field picture. In MPEG 2, like In MPEG 1 , there are 3 types of picture: I. B, and P. 

30 The coding method for an 1 picture is largely similar to the method in MPEG 1 . For a P picture the prediction direc- 
tion is the same as in MPEG 1 except that for a frame picture, a choice can be made among 3 prediction processes for 
the motion compensation prediction mode: frame prediction, field prediction or dual prime prediction. In addition, for 
field pictures, a choice can be made among: frame prediction, 16 X 8MC prediction or dual prediction. Dual prime pre- 
diction is a variation of field prediction in which prediction for 2 fields included in a frame is carried out using an identical 

35 parity field and a different parity f ietel. When decoding, this produces a total of 4 (in the case of frame pictures) or 2 (in 
the case of field pictures) motion vectors. 

The prediction direction for B pictures is also tiie same as that in MPEG 1 but for a frame picture a choice can be 
made between 2 prediction processes for the motion compensation prediction mode: frame prediction or fleki predic- 
tion. In addition, for field pictures a choice can be made between field prediction or16 x 8 MC prediction. 

40 Thus. 2 motion vectors are generated fbr the single prediction direction only if field prediction is carried out for a 
frame picture, or if 1 6 x 8 MC prediction is carried out fbr a field picture. 

[Operation] 

45 Differences In operation of the apparatus in the second embodiment will new be explained. 

[1] Normal Decoding Operation 

When the target of tiie decoding is an inter-coded macro blocK tiie motion vector selection portion 14 latches the 
50 motion compensation prediction mode in addition to the motion vector and prediction direction information sent through 
. the parameter bus 42. Reference is made to this motion compensation prediction mode during tiie following processes. '^i,. 

[2] Decoding Operation when Error Is Detected 

55 (1 ) Concealment Processing 

In the present emkxxliment tiie prediction picture, motion vector and motion compensation prediction mode used 
for concealing are discussed below for each type of pictura 
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1. (pictures 

In principle, the same as in the second embodiment. However, with regard to the motion compensation prediction 
mode, if the picture is a frame picture then frame prediction is used, if it is a field picture then field prediction is used. 
5 This fixed process is only effective for concealment and when decoding is actually carried out. the motion compensation 
prediction mode specified at coding must be used. 

2. P pictures 

10 The estimated motion vector and prediction picture are the same as in the second embodiment but when there are 
two nfx>tion vectors, the first to be decoded is used. Witii regard to the motion conrpensation prediction mode, as with 
the above-mentioned I picture, if the picture is a frame picture then frame prediction is used, if it is a field picture tiien 
field prediction is used. Also as with the above-mentioned I picture, this fixed process is effective only for concealment. 

15 3. B pictures 

The same as P pictures. 

The motion vector selection portion 14 complies with tiie above stipulations and outputs a final motion compensa- 
tion prediction mode in addition to a final motion vector 15 and a final prediction direction 51 . TTie motion vector selec- 

20 tion portion 1 4 in this embodiment differs in tiiat it also latches the picture structure and motion compensation prediction 
mode of the picture which includes tiie maao block being decoded. 

The motion compensation prediction mode used in concealment is not stored at the motion vector selection portion 
1 4 because it is always determined by the picture structure. When the error flag 8 is at 0, the decoded motion vector, 
prediction direction and motion compensation prediction mode are output as they are to the final motion vector deter- 

25 mining portion 30. When the error flag 8 is at 1 , the adequacy of the motion vector Is evaluated based on tfie on-screen 
position information of that macro block, tiie estimated motion vector 20, read from tiie motion vector butter 26, and on 
the estimated prediction direction 69. If it judged to be inadequate, the vector value of the final motion vector 15 is set 
at 0 and tiie final prediction direction 51 is output as fonward. If It is judged to be adequate, then the final motton vector 
1 5. tiie final estimated prediction directksn 51 and tiie final compensation prediction mode which have been determined 

30 based on the picture sti'ucture of the picture to which the estimated motion vector 20 and tiie estimated prediction direc- 
tion 69 and the macro block belong are output Thereafter tiie process Is basically similar to tiiat in the second embod- 
iment. 

(2) Judging and Filtering of Contaminated Regions 

35 

As In tiie second embodiment, registering of contaminated regions is canried out only for I and P pictures. The con- 
figuration of contaminated region judging portion 16 in tiiis embodiment is largely the same as that in the second 
embodiment. It differs in that contaminated region registering and judging portion 31 also makes reference to motion 
compensation prediction mode In addition to those references descrit»ed In tiie second embodiment 
40 When the enror flag 8 is at 1 , tiie macro block being decoded is regarded as a contaminated region in the contam- 
inated region registering and judging portion 31 and its on-screen position Information is registered in the contaminated 
region memory for use with decoding pictures 36. 

Alternatively, when the error flag 8 is at 0, judging is only carried out for P and B pictures. In MPEG 2 tfiere are 3 
possible motion compensation prediction modes for the 2 picture structures. Of ttiese. in frame prediction for frame pic- 
45 tures and f ieki prediction and dual prime prediction for f lekJ pictures, tiiere will be 1 or 2 prediction images and process- 
ing will be the same as tiiat shown In Fig. 16. 

Othenwise, In field prediction and dual prime prediction for frame pictures and 1 6 x 8MC prediction for f iekJ pictures, 
there vtfill be 2 or 4 prediction Images. A judging method in the case where there are 4 prediction images will now be 
explained. 

50 Fig. 1 7 Is a figure showing a judging method for contaminated region registering and judging portion 31 in the case 

where there are 4 prediction Images. This figure depicts the.case where fieki prediction Is applied to a frame picture. In.^.< *ri^'. 
- both the foHA^rd prediction picture and tiie reanrard prediction picture, prediction images exist for No. 1 field and No. 2 . 
field. The contaminated region distribution is first obtained for the fonvard prediction picture and ttie reanward prediction 
picture. Here SI and S2 are defined as: 

55 

- SI is the portion of tiie predicted Image region 140 fbr tiie No. 1 field in tiie fbnivard prediction picture which does 
not overlap with the contaminated region 38 of that picture 

- S2 is the portion of the predicted Image region 140 for tiie No. 1 field in the fonvard prediction picture which over- 
laps with the contaminated region 38 of tiiat picture 
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S3~8 are similarly defined as shown in the figure. Thus the contaminated region share P3 of the predicted image 
is defined to be 

P3«{S2/(S1 +S2) + S4/(S3 + S4) + S6/(S5 + S6) + S8/(S7 + S8))/4 
When P3 satisfies the condition below, that predicted image is regarded as contaminated: 

PS > TH (Equation 3) 

In only fbnward prediction 

P3 = {S2 /(SI + S2) + S4/(S3 + S4) }/2 

and In only of reanvard directional prediction 

P3 » {S6/(S5 + S6) + S8/(S7 + S8) }/2 

When judging has ended, a filtering process is carried out. In the present emtxxiiment a loop filter portion 20 oper- 
ates according to the picture structure. Fig. 18 is an internal configuration figure for the loop filter portion 20 in the third 
embodiment. The loop filter portion 20 consists of a blocking control portion 127 fbr controlling SW1 and SW2 in com- 
pliance with the picture structure and filtering Instruction flag 1 7 latched from the parameter bus 42. a field blocking por- 
tion 129 fbr rearranging a frame format predicted Image into field frame format, and a fitter portion 130 fbr filtering of 
both predicted image which has passed and by-passed the field blocking portion 129. The blocking control portion 127 
connects SW1 and SW2 to A only when the picture structure is frame and contaminated region flHering instruction flag 
17 is at 1. 

In this configuration if the picture structure is frame and contaminated region filtering instruction flag 1 7 is at 1, the 
field blocking portion 129 carries out data conversion as shown in the figure. Thereafter, filtering is executed. Conse- 
quently even if there is motion between fields when smoothing is carried out (for each white and black portion in the 
figure) any loss of this motion due to smoothing is averted. After filtering the f leM format can be returned to frame format 
by a inverse substituting drcurt (not shown In the figure). 

The following improvements or modifications to the present embodiment are also envisaged: 

1 . As in the first and second embodiments the configuration may be such that the registering of contaminated 
regions is deleted within a certain period. 

2. By utilizing a given combination of the apparatuses in embodiments 1-3 a motion image processing apparatus 
can be provided capable of decoding a bit stream in accordance with any one of the following: ITU-TH.261 stipula- 
tions. MPEG 1 video stipulations. MPEG 2 video stipulations. 

While there has been described what are at present considered to be preferred emtxxjiments of the invention, it will 
be understood that various modifications may be made to the present invention and it is intended that the appended 
claims cover all such modifications as fall witiiin ttie true spirit and scope of the invention. 

Claims 

1 . A motion image decoding method for decoding coded motion Image data using motion compensation prediction, 
said method comprising: 

an analysis process fbr analyzing motion image data at each processing unit image; 
a registering process for registering a region of a processing unit image as a contaminated region when an 
en^or has been detected during analysis of the processing unit image; 
r ^\ . -a determining process for determining a predicted image to be used for motion compensation prediction when 
r- a processing unit image is decoded; 

a judging process for judging if a determined predicted image is included in said contaminated region; and 
a smootiiing process for smoothing a predicted image when said predicted image has been judged to be 
included in said contaminated region. 

2. A motion image decoding method according to claim 1 . wherein 

said registering process registers not only a region of a processing unit image in which an error has been 
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detected but also region for which motion compensation prediction has been candied out using a predicted 
image which is included In a contaminated region. 

3. A motion image decoding apparatus for decoding coded motion image data using motion oonpensatlon prediction, 
said apparatus comprising: 

a bit stream analysis portion (200) for receiving motion Image data and analyzing each processing unit Image 
of said motion image data; 

a contaminated region registering portion (31) for registering a region of a processing unit image as a contam- 
inated region when an error has been detected during analysis of a processing unit image: 
a predicted image determining portion (14) for determining a predicted image to be used for motion compen- 
sation precfiction when a processing unit image is decoded; 

a contamination judging portion (16) for judging if a predicted image Is contaminated by comparing a region of 
a determined predicted image with said contaminated region; and 

a smoothing portion (20) for smoothing a predicted image when said predicted image has been judged to be 
included in said contaminated region. 

4. A motion Image decoding apparatus according to claim 3, wherein 

said predicted Image determining portion (1 4) carries out motion compensation prediction from the motion vec- 
tor for each processing unit Image and determines a predicted image for a processing unit image which has 
lost Its motion vector due to an error based on a motion vector of a processing unit image in close proximity. 

5. A motion image decoding apparatus according to claim 3 and 4, further comprising a picture counting portion (208) 
for counting the number of decoded pictures, wherein 

when the number of pictures counted has reached a designated value, said contaminated region registering 
portion (31) erases registration of the contaminated regfon. 

6. A motion Image decoding apparatus according to dalm 3 to 4, wherein 

said contaminated region registering portion (31) registers not only a region of a processing unit image in 
which an en'or has been detected but also region for which motion compensation prediction has been carried 
out using a predicted Image which Is included in a contaminated region. 

7. A motion image decoding apparatus according to claim 3 to 6, wherein 

said contamination judging portion (16) judges the existence of contamination from the level of overlap of the 
region of the said predicted Image and the said contaminated region. 

8. A motion Image decoding apparatus according to dalm 4 to 7, furtiier comprising 

a loss image extent determining portion (204) for determining, at a time when an error has been detected, not 
only the processing unit image in which the en^or has been detected but also the extent of the image region 
which, like tiie processing unit image, has iost its motion vector as a result of the en'or; wherein 
said contaminated region registering portion (31) registers tiie whole extent of this image region as a contam- 
inated region. 

9. A motion image decoding apparatus according to claim 3 to 8. furtiier comprising 

a contaminated region memory portion for use with decoding pictures (36) and a contaminated region memory. 
' ■ - portion for use with prediction pictures (37), 

such that when an error is detected in a picture presentiy being decoded, said contaminated region registering 
portion (31) registers the processing unit image in which the error has been detected in the contaminated 
region memory portion for use with decoding pictures (36), and which, when the decoding processing has pro- 
ceeded to tiie next picture, switohes the contaminated region memory portion for use with decoding pictures 
(36) with the contaminated region memory portion for use witti prediction pictures (37). 

10. A motion image decoding apparatus according to daim 3 to 9. furtiier comprising: 
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a picture configuration judging portion (206) for judging whether the configuration of a picture is in frame format 
or in field format; 

a picture configuration converting portion (129) for converting a picture configuration which is in frame format 
to a configuration in field format; wherein 

when a picture configuratran is in frame format, said smootiiing process is carried out only after the configura- 
tion has been converted to field format. 
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